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@ Image processing method and apparatus. 

(57) An image processing apparatus, as well as an 
image processing method, capable of execut- 
ing area separation accurately using a small 
block size includes a wavelet transforming cir- 
cuit for converting image data, which has en- 
tered from an image input circuit, into a 
plurality of band-limited image data, an edge 
detecting circuit for detecting edge information 
in the image data based upon the band-limited 
image data, a character-area detecting circuit 
for detecting character areas in the image data 
from the distribution of the edge information, 
and an expansion/contraction circuit for unify- 
ing the character areas based upon the charac- 
ter-area information detected. 
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BACKGROUND OF THE INVENTION 



SUMMARY OF THE INVENTION 



Field of the Invention: 

This invention relates to an image processing 5 
method and apparatus. More particularly, the inven- 
tion relates to an image processing method and ap- 
paratus, for use in a facsimile machine, copier or the 
like, through which an input image is separated into 
a variety of images, as in the manner of a character 10 
area and a non-character area, and processing such 
as encoding is executed for each area into which the 
image has been separated. 



Description of the Related Art: 



15 



A method that relies upon the distribution of high- 
frequency components contained in an image often 
is used as a method of separating the image into 
character and non-character areas. This method is as 20 
follows: 

(1) the image is separated into m x n blocks; 

(2) edge blocks are detected as by a Laplacian 
operation; 

(3) the edge blocks are counted; and 25 

(4) based upon the relationship between the 
number n of edge blocks and a threshold value T, 
the m x n blocks are discriminated in the following 
manner, a block is a character block If n > T holds, 

and a block is a non-character image block if n £ 30 
T holds. 

Further, a color facsimile apparatus in which 
character areas and non-character areas are encod- 
ed by an appropriate method has been proposed [see 
Serial No. 631145 (filed on December 20, 1990) and 3$ 
Serial No. 651030 (February 5, 1991)). 

However, a problem encountered in the prior art 
described above is that when the size of the m x n 
blocks is small, it is difficult to distinguish between an 
edge portion of a character area and an edge portion 40 
of a non-character area. This means that when it is at- 
tempted to perform precise discrimination, a fairly 
large block size is required. This is disadvantageous 
in that hardware of larger scale becomes necessary. 

Furthermore, when conventional block process- 45 
ing of the kind described above is executed, edge de- 
tection is difficult if an edge is present at the bound- 
ary between one cell of m x n blocks and an adjacent 
cell of m x n blocks. A problem that arises is that a 
character area may be judged as being a norvcharac- so 
ter area. 

Another problem in the aforementioned prior art 
is that the portion underlying a character area (name- 
ly a color or pattern underlying the character area) 
vanishes, as a result of which part of the image infor- 55 
mat ion is lost 



An object of the present invention is to make it 
possible to accurately identify the characteristics of 
an input image. 

According to the present invention, the foregoing 
object is attained by providing an image processing 
apparatus comprising input means for entering image 
data, conversion means for converting the image 
data into a plurality of band-limited image data by us- 
ing a band-pass filter, and discrimination means for 
discriminating a characteristic of an image, which is 
represented by the image data, on the basis of at least 
one of the plurality of band-limited image data. 

Another object of the present invention is to make 
it possible to realize such identification through a 
simple arrangement 

A further object of the present invention is to 
make effective utilization of band-limited image data. 

Still another object of the present invention is to 
provide an efficient coding method using results ob- 
tained from the aforementioned identification. 

Other features and advantages of the present in- 
vention will be apparent from the following descrip- 
tion taken in conjunction with the accompanying 
drawings, in which like reference characters desig- 
nate the same or similar parts throughout the figures 
thereof. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram illustrating the construc- 
tion of an image processing apparatus according 
to an embodiment of the present invention; 
Fig. 2 is a block diagram illustrating the construc- 
tion of a wavelet transforming circuit shown in 
Fig. 1; 

Fig. 3 is a diagram in which the hierarchical output 
of the wavelet transforming circuit of Fig. 2 is ex- 
pressed in frequency space; 
Fig. 4 is a block diagram illustrating the construc- 
tion of an edge detecting circuit shown in Fig. 1; 
Fig. 5 is a diagram for describing an example of 
the operation of a character-area detecting circuit 
shown in Fig. 1; 

Fig. 6 is a block diagram illustrating the construc- 
tion of the character-area detecting circuit shown 
in Fig. 1; 

Fig. 7 is a flowchart illustrating the operating pro- 
cedure of an expansion/contraction circuit shown 
in Fig. 1; 

Figs. 8A, 8B are flowcharts illustrating the oper- 
ating procedure of the expansion/contraction cir- 
cuit shown in Fig. 1; 

Fig. 9 is a block diagram illustrating the construc- 
tion of an image processing apparatus according 
to a second embodiment of the present invention; 
Fig. 10 is a block diagram illustrating the corv 
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struct ion of an encoder shown in Fig. 9; 
Fig. 11 is a flowchart illustrating an edge-pixel 
detecting procedure of a second edge detecting 
circuit shown in Fig. 10; 

Fig. 12 is a diagram showing an example of the 5 
relat ionship between a pixel of interest X and per- 
ipheral pixels; 

Fig. 13 is a block diagram illustrating the con- 
struction of a color detecting circuit shown in Fig. 
10; 10 
Fig. 14 is a diagram showing an example of the 
relationship between the inputs and output of a 
LUT shown in Fig. 13; 

Fig. 15 is a block diagram illustrating the con- 
struction of a color-character discriminating cir- 75 
curt shown in Fig. 10; 

Fig. 16 is a block diagram illustrating the con- 
struction of a binary-series converting circuit 
shown in Fig. 10; 

Fig. 17 is a diagram showing an example of bina- 20 
ry-series data outputted by the binary-series 
converting circuit of Fig. 16; 
Fig. 18 is a block diagram illustrating the con- 
struction of a color-character removal circuit 
shown in Fig. 10; 25 
Figs. 19A, 1 9B t 1 9C and 1 9D are diagrams for de- 
scribing the operation of the color-character re- 
moval circuit shown in Fig. 18; 
Fig. 20 is a block diagram showing the construc- 
tion of an orthogonal-transformation encoding 30 
circuit shown in Fig. 10; 

Fig. 21 is a block diagram illustrating the con- 
struction of a decoder shown in Fig. 9; and 
Fig. 22 is a block diagram illustrating the con- 
struction of an image memory apparatus accord- 35 
ing to another embodiment of the present inven- 
tion. 

DESCRIPTION OF THE PREFERRED 

EMBODIMENTS 40 

An image processing apparatus according to em- 
bodiments of the present invention will now be descri- 
bed in detail with reference to the drawings. 

45 

(First Embodiment] 

Fig. 1 is a block diagram illustrating the arrange- 
ment of the first embodiment 

As shown in Fig. 1, the image processing appa- so 
ratus according to this embodiment includes an im- 
age input circuit 1 for entering, in line units, lumi- 
nance-image data of an image to be separated into 
areas. In a case where RGB data or YMC data, etc., 
has been applied to the image input circuit 1 , the latter 55 
converts the input data into luminance-image data by 
a well-known method and then outputs the resulting 
data. 



The image input circuit is connected to a wavelet 
transforming circuit 2. Though the details will be de- 
scribed later, the wavelet transforming circuit 2 ap- 
plies a two-dimensional wavelet transformation to 
the image data that has entered from the image input 
circuit 1, thereby delivering a third hierarchical layer, 
for example, of image data. 

The wavelet transforming circuit 2 is connected 
to an edge detecting circuit 3. Though the details will 
be described later, three items of band image data, 
from which band image data representing the lowest 
frequency component has been eliminated, enter the 
edge detecting circuit 3 from the third hierarchical lay- 
er of band image data resulting from the conversion 
performed by the wavelet transforming circuit 2. The 
edge detecting circuit 3 detects edge pixels in this in- 
put data. 

A character-area detecting circuit 4 detects mac- 
ro-character areas based upon the band image data 
applied thereto by the edge detecting circuit 3. The 
details will be described later. 

The character-area detecting circuit 4 is connect- 
ed to an expansion/contraction circuit 5. Though the 
details will be described later, the expansion/contrac- 
tion circuit 5 applies dot expansion/contraction proc- 
essing to character areas, which have been detected 
by the character-area detecting circuit 4, in such a 
manner that scattered character areas will be consoli- 
dated as much as possible. The resulting area dis- 
crimination data is outputted. 

Since a third layer of image data is used in this 
embodiment, the area discrimination data obtained 
by the foregoing processing is made a size that is one- 
eighth that of the input image data in both the main- 
scan direction (input-line direction) and sub-scan di- 
rection. 

Fig. 2 is a block diagram illustrating the construc- 
tion of the wavelet transforming circuit 2. This circuit 
is a four-tap filter an example of the filter coefficients 
of which are shown in Table 1 below. In this embodi- 
ment, the number of taps of the filter and the number 
of filter coefficients are not limited to those indicated 
in the table. 



L 


H 


0.482963 


-0.129410 


0.836516 


0.224144 


0.224144 


0.836516 


-0.129410 


0.482963 


TABLE 1 



In Fig. 2, the numerals 11, 15, 19, 23, 27, 31, 35, 
39 indicated by "L* represent low-pass filters (here- 
inafter referred to as "LPF's). These filters apply one- 
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dimensional low-pass filtering to the entered image 
data in the main-scan direction. 

The numerals 9, 13, 17, 21, 25, 29. 33, 37, 41 in- 
dicated by "H" represent high-pass filters (hereinafter 
referred to as p HPF*s) . These filters apply one-di- 5 
mensional high-pass filtering to the entered image 
data in the main-scan direction. 

Further, the numerals 8. 10, 12, 14, 16, 18, 20. 22. 
24, 26, 28, 30, 32. 34. 36. 38. 40, 42 indicated by m i m 
represent sampling circuits for sampling, at a ratio of io 
2:1 in the main-scan direction, the image data that 
has entered from the immediately preceding LPF or 
HPF. 

More specifically, the wavelet transforming cir- 
cuit 2 subdivides entered image data 500 into ten is 
bands of image data 503 - 512 by filtering. According 
to this embodiment, image data 512 having the high- 
est frequency band is obtained by passing the en- 
tered image data 500 through the circuitry HPF 9 
sampling circuit 10 -> HPF 17 -» sampling circuit 18, 20 
and image data 503 having the lowest frequency 
band is obtained by passing the entered image data 
500 through the circuitry LPF 7 -> sampling circuit 8 
-> LPF 11 -> sampling circuit 12 -> LPF 19 -> sam- 
pling circuit 20 -* LPF 23 -» sampling circuit 24 -> LPF 25 
31 -> sampling circuit 32 LPF 35 -» sampling circuit 
36. 

Fig. 3 is a diagram in which the hierarchical output 
of the wavelet transforming circuit 2 is represented in 
frequency space. 30 

As shown in Fig. 3, the higher the hierarchical lay- 
er of the image data (for example, image data 501 has 
a higher hierarchical layer than image data 500; im- 
age data 502 has a hierarchical layer than the image 
data 501; and image data 503 has a hierarchical layer 35 
than the image data 502). the broader the information 
possessed by the image. 

Accordingly, in area subdivision in which broad 
processing is necessary in view of macro areas, this 
processing can be achieved by using an image hav- 40 
ing a higher hierarchy. In particular, in a case where 
image data is subdivided into character portions and 
non-character portions, the character portions, for 
example, are simply binarized and then subjected to 
arithmetic encoding and the non-character portions 45 
are subjected to DCT (direct cosine transform) en- 
coding, good results can be obtained by using image 
data 503 - 506. The reason for this is that a DCT gen- 
erally is performed in units of 8 x 8 pixels, with one 
pixel of image data 503 - 506 corresponding to 8 x so 
8 pixels of the original image. 

Further, image data 504 - 506, which represents 
the high-frequency components of the image data 
502, indicates the size of an edge. Accordingly, in this 
embodiment, edge detection is carried out using 55 
three-band image data of the image data 504 - 506. 

The arrangement depicted in Fig. 2 outputs ten 
bands of image data 503 - 512. However, since three 



bands of image data are used for edge detection in 
this embodiment, as mentioned above, the wavelet 
transforming circuit 2 need only have a construction 
for outputting the image data 504 - 506. 

Further, according to this embodiment the be- 
ginning and end of processing, which represent a 
problem in filtering, are dealt with by repeating the in- 
put image data periodically. However, this can be 
dealt with by doubling back the image, e.g., by adopt- 
ing a mirror image, at the beginning and end portions 
of the image data. 

Fig. 4 is a block diagram illustrating the construc- 
tion of the edge detecting circuit 3. 

As shown in Fig. 4 , the edge detect ing circuit 3 in- 
cludes an absolute value circuit 61 for outputting the 
absolute values of image data 504 - 506 entering 
from the wavelet transforming circuit 2, an adder 62 
for summing the pixel data corresponding to the three 
items of image data that have entered from the abso- 
lute-value circuit 61, a normalizing circuit 63 for nor- 
malizing, to a certain range (e.g., 0 ~ 255), the sum- 
med data that has entered from the adder 62, and a 
binarizing circuit 64 for binarizing the normalized 
data, which has entered from the normalizing circuit 
63, at a certain threshold value and outputting the re- 
sults. Through the binarizing threshold value is made 
40, for example, this does not impose a limitation 
upon the invention; an appropriate threshold value 
should be set in conformity with the distribution of the 
image data. 

More specifically, the edge detecting circuit 3 ex- 
tracts only pixels (namely an edge) exhibiting a 
somewhat abrupt change in tone on the basis of the 
image data 504 - 506. For example, an edge pixel is 
outputted as a "1" and a non-edge pixel is outputted 
as a "0*. 

It is also possible to find an edge from the image 
data 503 without using the image data 504 - 506. In 
such case, if the edge detecting circuit 3 is so con- 
structed as to subject the image data 503 to a Lapla- 
cian operation or the like to extract an edge and then 
binarize the results upon taking the absolute value, 
then binary data approximately the same as that pro- 
vided by the arrangement of Fig. 4 can be outputted. 

The character-area detecting circuit 4 detects 
character areas based upon the binary data output- 
ted by the edge detecting circuit 3. 

Fig. 5 is a diagram for describing the operation of 
the character-area detecting circuit 4. The center of 
the diagram illustrates a pixel of interest D. 

According to this embodiment, a character area 
is detected based upon 9x9 pixels 601 the center of 
which is the pixel of interest D. The 9x9 pixels 601 
are subdivided into blocks 602 of 3 x 3 pixels each. A 
number of edges E representing how many edges are 
present in a block, and a number of blocks NB repre- 
senting how many blocks for which the edge count E 
exceeds a threshold value ET are present, are count- 
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ed block by block, If the relation NUMBER OF 
BLOCKS NB £ 4 holds, it is judged that the pixel of 
interest D is part of a character area. 

Fig. 6 is a block diagram illustrating the construc- 
tion of the character-area detecting circuit 4. 5 

As shown in Fig. 6, the character-area detecting 
circuit 4 includes line memories 125- 133 for succes- 
sively storing one line of the binary data that has en- 
tered from the edge detecting circuit 3. Shift registers 
134 - 142 for successively storing the binary data io 
outputted by the corresponding line memories. Each 
shift register holds nine pixels of binary data. In other 
words, the shift registers 134 - 142 hold binary data 
representing the 9 x 9 pixels 601 shown in Fig. 5. 

Numerals 143 - 151 denote LUTs, each of which 15 
is constituted by a ROM or the like, having address 
terminals to which nine bits of data enter f rom prescri- 
bed bits of corresponding shift registers. Each LUT 
delivers a value, namely the number of "1"s (the num- 
ber of edges E) conforming to this nine-bit data, from 20 
its output terminal. The LUT (a) 143 outputs the num- 
ber of edges E1 in the 3 x 3 pixels 602. namely the pix- 
els 1a, 1b, 1c, 2a, 2b, 2c, 3a, 3b, 3c. Similarly, each 
of the LUTs 144 - 151 outputs a number of edges En 
present in the corresponding nine pixels. 25 

Numerals 152 - 160 denote comparators. The 
comparator (a) 152 compares the edge number E1, 
which has entered from the LUT (1) 143, with the 
threshold value ET and outputs a "1" if E1 > ET holds 
anda"0'ifE1 ^ET holds. Similarly, the comparators 30 
153 - 160 compare the edge numbers from the cor- 
responding LUTs with the threshold value ET and out- 
puts a "1* if En > ET holds and a "0" if En £ ET holds. 

Numeral 161 denotes a LUT (j). which is consti- 
tuted by a ROM, for example. The LUT 161 delivers 35 
a value, namely the number of "1*s (the number of 
blocks NB) conforming to the nine-bit data that has 
entered its address terminals from the comparators 
152 - 160. 

Numeral 162 denotes a comparator for compar- ao 
ing the number of blocks NB that has entered from the 
LUT (j) 161 with the criterion "4" on the basis of which 
an area is judged to be a character area. The compar- 
ator 161 outputs "1" is NB £ 4 holds and m 0 m if NB < 
4 holds. 45 

It should be noted that the operation of each block 
shown in Fig. 6 is synchronized by a pixel clock VCLK. 

Thus, the character-area detecting circuit 4 out- 
puts a character-area pixel as "1* and a non-charao 
ter-area pixel as "0". 50 

Though threshold value ET = 1 in this embodi- 
ment, any appropriate value may be used in depend- 
ence upon the image processed. 

Further, according to this embodiment, block 
number B £ 4 is adopted as the criterion for judging 55 
that an area is a character area. The reason is that 
there are many cases in which B < 4 holds, as in the 
case of a contour line of a non-character area in a pho- 



tograph or the like. In this way the contour line of a 
non-character area in a photograph or the like is 
avoided as being taken as part of a character area. 
Furthermore, in this embodiment, the criterion for 
judging a character area is not limited to B £ 4; other 
suitable values may be used in dependence upon the 
image processed. 

The judgment made by the character-area de- 
tecting circuit 4 is sent to the expansion/contraction 
circuit 5. The latter executes processing for joining 
and consolidating pixels, especially those that are 
scattered, judged as forming a character area. This is 
processing for roughly separating the entered image 
data into character and non-character areas. The 
processing comprises the two steps described below. 

The first step of this processing is expansion 
processing in which reference is made to, say, the 
eight pixels surrounding the pixel of interest to deter- 
mine if there is even one black pixel (a "1" pixel) 
among them. If there is, the pixel of interest is 
changed to black. The second step is contraction 
processing in which reference is made to, say, the 
eight pixels surrounding the pixel of interest to deter- 
mine if there is even one white pixel (a "0* pixel) 
among them. If there is, the pixel of interest is 
changed to white. 

More specifically, the expansion/contraction cir- 
cuit 5 subjects the binary data that has entered from 
the character-area detecting circuit 4 first to expan- 
sion processing ml times, then to contraction proc- 
essing n times and finally to expansion processing 
m2 times. 

The initial expansion processing is for the pur- 
pose of joining and consolidating scattered character 
pixels. Character pixels thus joined are not caused to 
scatter again by the subsequent contraction process- 
ing. 

Ordinarily, the expansion processing operation 
and the contraction processing operation are each 
executed several times. Isolated pixels that are not 
caused to join up with other pixels by the expansion 
processing remain as isolated pixels also when con- 
traction processing is executed. Many of such isolat- 
ed pixels reside in a non-character area such as a 
photograph or the like. If these isolated pixels are 
processed in the order of expansion • contraction - 
expansion as in this embodiment, then will be elim- 
inated by the contraction processing. This makes it 
possible to reduce erroneous judgment 

In this embodiment, ml, m2, n, which represent 
the numbers of times expansion/contraction proc- 
essing is executed, are set to 4, 1, 5, respectively. 
However, it will suffice if these values are integers 
that satisfy the following equation: 
n = ml + m2 (1) 

The expansion/contraction circuit 5, which is con- 
stituted by an image memory (not shown) and a sin- 
gle-chip CPU (not shown), executes the foregoing 
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processing in accordance with a program stored in a 
ROM incorporated within the CPU. The image mem- 
ory stores the binary data, which has been outputted 
by the character-area detecting circuit 4, at the cor- 
responding pixel positions. 5 

Figs. 7 and 8 are flowcharts illustrating the op- 
erating procedure of the expansion/contraction cir- 
cuit 5. 

According to this embodiment, as illustrated in 
Figs. 7 and 8, a variable j is set to 0 at step S1, ex- to 
pansion processing is executed at step S2 and the va- 
riable j is incremented at step S3. 

Next, the variable j and the processing count ml 
are compared at step S4. The program returns to step 
S2 if j < ml holds and proceeds to step S5 if j £ ml ts 
holds. 

If j £ ml holds, the variable j is set to 0 again at 
step S5. contraction processing is executed at step 
S6 and the variable j is incremented at step S7. 

Next, the variable j is compared with the process- 20 
ing count n at step S8. The program returns to step 
S6 if j < n holds and proceeds to step S9 if j £ n holds. 

The variable j is set to 0 again at step S9 if j £ n 
holds, expansion processing is executed at step S10 
and the variable j is incremented at step S11. 25 

The variable j and the processing count m2 are 
compared at step S12. The program returns to step 
S10 if j < m2 holds and processing is terminated if j 
£ m2 holds. 

Fig. 8A is a flowchart illustrating the expansion do 
processing procedure. 

According to this embodiment, as shown in Fig. 
8A, a pixel of interest D is set at step S21 and the val- 
ue of the pixel of interest D is judged at step S22. The 
program proceeds to step S23 if D = p 0" holds and 35 
jumps to step S25 if D = T holds. 

In case of D = "0", it is determined at step S23 
whether the eight peripheral pixels contains a black 
pixel (a "1" pixel). The program proceeds to step S24 
if there is a black pixel and to jumps to step S25 in the 40 
absence of a black pixel. 

In case of a black pixel, the pixel of interest D is 
set to "1" (black) at step S24. 

Next, it is determined at step S25 whether the 
processing of all pixels has ended or not The program 45 
proceeds to step S21 if processing has not ended and 
returns to the main routine of Fig. 7 if processing has 
ended. 

Fig. 8B is a flowchart illustrating the contraction 
processing procedure. 50 

According to this embodiment, as shown in Fig. 
8B, a pixel of interest D is set at step S61 and the val- 
ue of the pixel of interest D is judged at step S62. The 
program proceeds to step S63 if D = V holds and 
jumps to step S65 if D = "0* holds. 55 

In case of D = "1", it is determined at step S63 
whether the eight peripheral pixels contains a white 
pixel (a *0" pixel). The program proceeds to step S64 



if there is a white pixel and to jumps to step S65 in the 
absence of a white pixel. 

In case of a white pixel, the pixel of interest D is 
set to "0" (white) at step S64. 

Next, it is determined at step S65 whether the 
processing of all pixels has ended or not. The program 
proceeds to step S61 if processing has not ended and 
returns to the main routine of Fig. 7 if processing has 
ended. 

Thus, the expansion/contraction circuit 5 outputs 
binary image data, in which a pixel of a character area 
is made T (black), as the result of discriminating a 
character area. 

In accordance with this embodiment, as descri- 
bed above, image data having frequency compo- 
nents of a third hierarchical layer following a wavelet 
transformation is used in area discrimination. As 8 re- 
sult, highly accurate character-area discrimination 
can be performed in units of 8 x 8 pixels of the original 
image. It is particularly beneficial when the results of 
discrimination are utilized in standard compression 
(so-called ADCT) of a JPEG color image. 

Further, in accordance with this embodiment, 
band-limited image data per se f from which image 
data having the lowest frequency band following wa- 
velet transformation has been removed, represents 
the edge quantity (the high-frequency components). 
Therefore, processing such as a Laplacian operation 
need not be applied anew, processing speed is raised 
and circuitry is simplif ied. Furthermore, since extrac- 
tion of a character area from a local edge is carried 
out by scanning a window in pixel units rather than by 
executing block processing, erroneous judgments at 
block boundaries, which was a problem in the prior 
art, can be reduced in number. 

Further, in accordance with this embodiment, 
isolated pixels are eliminated by executing process- 
ing in the order of expansion - contraction - expansion 
when joining up character areas as post-processing. 
This makes it possible to improve the accuracy with 
which character and non-character areas are judged. 

[Second Embodiment] 

A second embodiment of the present invention 
will now be described. The second embodiment deals 
with an image communication apparatus such as a 
color facsimile machine that incorporates the first 
embodiment in the form of an area separating unit 
Components similar to those of the first embodiment 
are designated by like reference characters in the 
second embodiment and need not be described in de- 
tail again. 

Fig. 9 is a block diagram illustrating the construc- 
tion of the second embodiment. 

As shown in Fig. 9, an image reader 101 scans 
the image of an original by a CCD line sensor or the 
like and outputs eight-bit image data in each of the 
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colors R, G, B. An area separating unit 102, to which 
the image data from the image reader 101 is applied, 
separates the image data into character and non- 
character areas. Though the details will be described 
later, an encoder 103 encodes the image data, which 5 
enters from the image reader 101, on the basis of the 
data that has entered from the area separating unit 
102, and sends the encoded data out on a communi- 
cation line or the like. Though the details will be de- 
scribed later, a decoder 104 decodes encoded data, w 
which enters from a communication line or the like, 
into image data. An image output unit 105 outputs an 
image from the image data that enters from the de- 
coder 104. The image output unit 105 is constituted 
by a page printer such as a laser-beam printer, a serial 15 
printer such as an ink-jet printer or a display such as 
a CRT. 



[Encoder 103] 



20 



Fig. 10 is a block diagram illustrating the con- 
struction of the encoder 103. 

The encoder 103 includes a second edge detect- 
ing circuit 251 which, as will be described in detail be- 
low, detects edge pixels in the image data that has 25 
entered from the image reader 101 , a color detecting 
circuit 252 which, as will be described in detail below, 
detects prescribed color pixels in the image data that 
has entered from the image reader 101, and a color- 
character discriminating circuit 253 which, as will be 30 
described in detail below, is provided with the charac- 
ter-area data from the area separating unit 102, the 
results of edge-pixel detection from the second edge 
detecting circuit 251 and the results of prescribed col- 
or-pixel detection from the color detecting circuit 252, 35 
for detecting an edge and a pixel of a prescribed color 
contained in a character area. In other words, the dis- 
criminating circuit 253 detects a pixel of a color char- 
acter. 

The encoder 1 03 further includes a binary-series 40 
converting circuit which, as will be described in detail 
below, converts the color-character data from the col- 
or detecting circuit 252 into binary-series data, which 
is suitable for dynamic arithmetic coding, in depend- 
ence upon the results of discrimination from the col- 45 
or-character discriminating circuit 253, and an arith- 
metic encoding circuit 255 for dynamically arithmeti- 
cally encoding the binary-series data that has entered 
from the binary-series converting circuit 254. It 
should be noted that the encoding method of the ar- so 
ithmetic encoding circuit 255 and the construction of 
this circuit are as illustrated in the specification of 
Japanese Patent Application Laid-Open (KOKAI) No. 
2-65372 (USP 5.136,396). 

Further, as will be described in detail below, a 55 
color-character removal circuit 256 replaces a pixel in 
the image data from the image reader 101 that the 
color-character discriminating circuit has judged to be 



a color-character pixel with, say, the average value of 
the pixel block containing this pixel. An orthogonal- 
transformation encoding circuit 257 encodes the im- 
age data from the color-character removal circuit 256 
in accordance with the aforementioned ADCT meth- 
od, by way of example. A transmitting circuit 258 joins 
the encoded data that has entered from the arithmetic 
encoding circuit 255 and the orthogonal-transforma- 
tion coding circuit 257 and sends the result out on a 
communication line, tt should be noted that the trans- 
mitting circuit 258 first transmits the code of a color 
character and then transmits the codes of signals Y, 
Cr\ Cb' field-sequentially. However, a flag indicating 
the contents of each code is transmitted before the 
code. Further, the transmitting circuit 258 stores the 
codes in memory temporarily, thereby compensating 
for a time shift conforming to the transmission se- 
quence of the codes. 

The reason for detecting color characters in char- 
acter areas separated by the area separating unit 102 
in the coder 103 will now be described. 

The area separating unit 102 separates character 
areas from image data irrespective of what underlies 
these areas (e.g., light background colors, patterns, 
etc.). Therefore, when a pixels judged as being con- 
tained in a character area by the area separating unit 
102 is binarized as is, the underlying background is 
lost 

Accordingly, in this embodiment, a color charac- 
ter discriminated by the color- character discriminat- 
ing circuit 253 is removed from the image data by the 
color-character removal circuit 256, and the portion 
from which the color character has been removed is 
compensated for by using the average value of the 
surrounding pixels, thereby forming the underlying 
background. This is then encoded by the orthogonal- 
transformation encoding circuit 257. 

The second edge detecting circuit 251 will now 
be described. 

Fig. 11 is a flowchart illustrating the procedure of 
edge pixel detection performed by the second edge 
detecting circuit 251. 

The second edge detecting circuit 251 comprises 
an image memory (not shown), a single-chip CPU 
(not shown), etc., and executes the procedure of Fig. 
11 in accordance with a program stored in a ROM in- 
corporated within the CPU. 

In accordance with this embodiment, as shown in 
Fig. 11, a pixel of interest X is set at step S31 and a 
peripheral pixel E is selected at step S32. The periph- 
eral pixel E selected may be any one of pixels A - D 
shown in Fig. 12. 

Next, the operation indicated by the following 
equation is executed at step S33: 

S = V{(Xr - Er) 2 + (Xg - Eg) 2 + (Xb - Eb) 2 } (2) 
where Xr, En R data 
Xg, Eg: G data 



7 



13 



EP 0 611 051 A1 



14 



Xb. Eb: B data 

This is followed by step S34, at which the result 
5 of calculation and a threshold value TH1 are com- 
pared. The program proceeds to step S35 is S ^ TH1 
holds and to step S37 if S > TH1 holds. 5 

If S > TH1 holds, T is outputted at step S37. 
Next, it is determined at step S38 whether detection 
processing regarding all pixels has been executed. 
The program returns to step S31 if there are unpro- 
cessed pixels and the procedure is terminated if all 10 
pixels have been processed. 

Thus, the second edge detecting circuit 251 cal- 
culates the three dimensional spatial distance be- 
tween the pixel of interest X and the peripheral pixels 
A - D in accordance with Equation (2). In a case 75 
where the result S exceeds the threshold value TH1 
(e.g., 100), it is judged that an edge is present be- 
tween the pixel of interest X and the peripheral pixels 
A - D. If it is judged that there is even one edge be- 
tween the pixel of interest X and the peripheral pixels 20 
A ~ D, T (an edge pixel) is outputted as the result 
of judging the pixel of interest X. In accordance with 
this method of detection, it is possible to detect color 
edges having different hues and saturations even if 
brightness is the same. This is very effective in terms 25 
of detecting color characters. 

In addition to discriminating edges pixel by pixel, 
the second edge detecting circuit 251 determines 
whether an edge pixel is contained in an 8 x 8 pixel 
block to be processed in the color-character removal 30 
circuit 256 and orthogonal transformation encoding 
circuit 257, described below, and outputs the result of 
this determination. Though Fig. 12 illustrates an ex- 
ample of peripheral pixels for the purpose of perform- 
ing edge detection, the embodiment is not limited to 35 
this example. For instance, eight peripheral pixels 
may be used and the operation of Equation (2) may 
be performed based upon the average value of per- 
ipheral pixels and the pixel of interest X. 

The color detecting circuit 252 will be described aq 
next. 

Fig. 13 is a block diagram illustrating the con- 
struction of the color detecting circuit 252. 

As shown in Fig. 13, the color detecting circuit 
252 includes comparators 351 - 353, 354 - 356, 4$ 
subtracters 357 - 359 and comparators 360 - 362. 
The comparator (a) 351, for example, compares input 
data R with a threshold value TH2 and outputs "1" if 
R < TH2 holds and "0" if R £ TH2 holds. Similarly, the 
comparator (b) 352 outputs "1" if G < TH2 holds and so 
•0" if G £ TH2 holds, and the comparator (c) 353 out- 
puts "1* if B < TH2 holds and "0" if B £ TH2 holds. 

The comparator (d) 354, for example, compares 
input data R with a threshold value TH3 and outputs 
T if R > TH3 holds and "0" if R s TH3 holds. Simi- 56 
larly, the comparator (e) 355 outputs "1* if G > TH3 
holds and "0" if G £ TH3 holds, and the comparator 
(0 356 outputs Tif B> TH3 holds and "0" if B £ TH3 



holds. 

The subtractor (a) 357 outputs an absolute value 
I R-G I of the difference between the input data R and 
G, the subtractor (b) 358 outputs an absolute value 
I G-B I of the difference between the input data G and 
B, and the subtractor (c) 359 outputs an absolute val- 
ue I B-R I of the difference between the input data B 
and R. 

The comparator (g) 360, for example, compares 
input data I R-G I with a threshold value TH4 and out- 
puts "I" if I R-G I <TH4 holds and "0" if I R-G I £TH4 
holds. Similarly, the comparator (h) 361 outputs "1" 
if I G-B I < TH4 holds and "0" if I G-B I £ TH4 holds, 
and the comparator (i) 362 outputs "1 * if | B-R | < TH4 
holds and "0" if | B-R | £ TH4 holds. 

Numeral 363 denotes a LUT, constituted by a 
ROM or the like, having an output terminal D for out- 
putting results of coior discrimination conforming to 
the results of comparison applied to address termi- 
nals A from the nine comparators mentioned above. 

Fig. 14 is a diagram illustrating an example of the 
relationship between the inputs and output of the LUT 
363. By way of example, a pixel for which 

G, B < TH2 andR>TH3and |G-B| < TH4 
holds is judged to be an R (red) pixel, and a pixel for 
which 

G < TH2 and R, B > TH3 and ! R - B ! < TH4 
holds is judged to be an M (magenta) pixel. The LUT 
363 outputs the results of detection as R, G. B data 
of one bit each, by way of example. Accordingly, the 
LUT 363 outputs "000" if a K (black) pixel is detected, 
"100" if an R (red) pixel is detected, "010" if a G 
(green) pixel is detected, "001* if a B (blue) pixel is de- 
tected, "011 * if a Y (yellow) pixel is detected, "101" if 
an M (magenta) pixel is detected, "110" if a C (cyan) 
pixel is detected and "111" (white) in case of a pixel 
that does not apply to any of these colors. 

When 50, 205, 30 are set as the above- 
mentioned threshold values TH2, TH3, TH4, respec- 
tively, excellent results of detection are obtained. 

The color-character discriminating circuit 253 will 
now be described. 

Fig. 15 is a block diagram illustrating the con- 
struction of the color-character discriminating circuit 
253. 

The color-character discriminating circuit 253 in- 
cludes a NAND gate 371 for output ting the NAND of 
each bit of the RGB data that has entered from the 
color detecting circuit 252, and an AND gate 372 for 
taking the AND of the results of area separation from 
the area separating unit 102, the results of edge de- 
tection from the second edge detecting circuit 251 
and the data from the NAND gate 371, and delivering 
the results of the AND operation as discrimination 
data S. 

Thus, the color-character discriminating circuit 
253 judges that a color-character pixel is a pixel in a 
block in which a pixel corresponding to an edge is 
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judged to exist by the second edge detecting circuit 
in an area judged to be a character area by the area 
separating unit 102, this pixel also being judged to be 
any of the K, R, G. B, C, M, Y pixels by the color de- 
tecting circuit 252. The circuit 253 outputs the dis- 5 
crimination data S = "1" in response to discrimination 
of this color-character pixel. 

The binary-series converting circuit 254 will be 
described next 

Fig. 16 is a block diagram illustrating the con- io 
struction of the binary-series converting circuit 254. 

The convert ing circuit 254 includes a LUT 91 con- 
stituted by a ROM, for example. If the discrimination 
data that has entered from the col or- character dis- 
criminating circuit 253 is T, the LUT 91 outputs bi- 15 
nary-series data of 1 - 7 bits, an example of which 
is illustrated in Fig. 17, corresponding to the R, G, B 
data of one bit each that has entered address termi- 
nals Afrom the color detecting circuit 252. The binary- 
series data is delivered from a data terminal D. 20 

The binary-series converting circuit 254 further 
includes a signal output unit 92 to which the binary- 
series data of 1 - 7 bits from the LUT 91 is applied. 
The signal output unit 92 delivers the binary-series 
data, as a binary-series output signal D, one bit at a 2S 
time in the form of a serial starting from the most sig- 
nificant bit (hereinafter referred to as the "MSB"). If 
the signal output unit 92 outputs "1" or seven "0"s. the 
output of one pixel is terminated and the next input 
data is received. Further, the signal output unit 92 out- 30 
puts also a three-bit signal Bt, for example, for indi- 
cating which bit of the binary-series data is currently 
being outputted. 

Thus, according to this embodiment, the R, G, B 
data each of one bit outputted by the color detecting 35 
circuit 252 is converted into a serial binary signal in 
dependence upon the results of discrimination per- 
formed by the color-character discriminating circuit 
253, after which this binary-series signal is encoded 
by the arithmetic encoding circuit 255. As a result, not 40 
only is each bit of the mutually correlated R, G, B data 
encoded individually but this encoding can be per- 
formed while preserving the hue correlation. More- 
over, in a case where encoding is performed while 
predicting the value of a pixel of interest, as in arith- 45 
metic encoding, prediction and encoding are not per- 
formed for every color component of RGB. Rather, 
prediction and encoding can be carried out using col- 
or information. This makes it possible to improve en- 
coding efficiency. 50 

Each color component of RGB representing the 
color of each pixel is represented by one item of data. 
Therefore, when decoding is performed, the RGB 
data of one pixel can be obtained at one time by de- 
coding one item of data. This makes it possible to rap- 55 
idly reproduce a color image. 

The color-character removal circuit 256 will now 
be described. 



Fig. 18 is a block diagram illustrating the con- 
struction of the color-character removal circuit 256. 

As shown in Fig. 18, the color-character removal 
circuit 256 includes a color detector 71, which has a 
construction substantially the same as that of the col- 
or detecting circuit 252, for detecting a color pixel and 
its peripheral pixels from the entered RGB data. Ac- 
cordingly, the threshold values TH2. TH3, TH4 of the 
comparator circuits differ from those of the color de- 
tecting circuit 252, e.g., these values are set to 120, 
130, 30, respectively, which are values that allow de- 
tection over a broader range. 

An OR gate 72 outputs the OR of the one-bit R, 
G, B items of data that have entered from the color de- 
tector 71, and an AND gate 73 outputs the AND be- 
tween the output of the OR gate 72 and the discrim- 
ination data S that has entered from the color-char- 
acter discriminating circuit 253. 

A selector (a) 74 outputs the data that has been 
applied to either a terminal A or a terminal B, depend- 
ing upon a selection signal applied to a selection ter- 
minal S from the AND gate 73. It should be noted that 
the selector (a) 74 outputs the pixel data from the im- 
age reader 101 if the selection signal is "0" and out- 
puts data (0) if the selection signal is T. 

An average value circuit 75 outputs the average 
value of pixels in, say, an 8 x 8 pixel block from the 
RGB data that has entered from the selector (a) 74. 
It should be noted that the pixel block size that deter- 
mines the pixel average value is made the same as 
the encoding block size of the orthogonal-transfor- 
mation encoding circuit 257, which is the next stage. 

A selector (b) 76 outputs the data that has been 
applied to either a terminal A or a terminal B, depend- 
ing upon a selection signal applied to a selection ter- 
minal S from the OR gate 72. It should be noted that 
the selector (b) 76 outputs the pixel data from the se- 
lector (a) 74 if the selection signal is *0" and the pixel 
average value from the average- value circuit 75 if the 
selection signal is "1 B . 

Figs. 19A - 19D are diagrams for describing an 
example of the operation of the color-character re- 
moval circuit 256. 

Fig. 19B shows an example of the image data 
that enters the selector (a) 74. This illustrates a 
change in black level in 8 case where the image de- 
picted in Fig. 19A has been read at line 903. 

As set forth above, the output of the OR gate 72 
is "1" if the pixel data exceeds the threshold value 
shown in Fig. 19B. If the discrimination data S from 
the color-character discriminating circuit 253 is "1" at 
the same time, then the output of the AND gate 73 
also becomes "1". Accordingly, the image data out- 
putted by the selector (a) 74 has data 901a corre- 
sponding to an image 901 removed from it, whereby 
the result shown in Fig. 19C is obtained. Further- 
more, the image data outputted by the selector (b) 76 
has the portion of the remaining data 901b of image 
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901 averaged, whereby the result shown in Fig. 19D 
is obtained. 

In the foregoing description, an example in which 
a color-character pixel is replaced by an average val- 
ue of pixels in, say, an 8 x 8 pixel block is described. 5 
However, this does not impose a limitation upon the 
embodiment. An arrangement can be adopted in 
which the color-character pixel is replaced by the val- 
ue that appears most frequently in the block or by the 
central value of the same block, which is obtained by w 
a median filter. 

The orthogonal-transformation encoding circuit 
257 will be described next. 

Fig. 20 is a block diagram illustrating the con- 
struction of the orthogonal-transformation encoding 15 
circuit 257. 

As shown in Fig. 20, the encoding circuit 257 in- 
cludes a pre-processor 81 for converting the RGB 
data from the color-character removal circuit 256 into 
a luminance signal Y and color-difference signals Cr, 20 
Cb, and a sampling unit 82 for outputting average val- 
ues Cr', Cb' of, say, every 2x2 pixel block of the sig- 
nals Cr, Cb that have entered from the pre-processor 
81. This makes use of the fact that the nature of the 
sense of sight is such that a deterioration in the color- 25 
difference signals Cr, Cb is more difficult to ascertain 
in comparison with that of the luminance signal Y. 

The encoding circuit 257 further includes an en- 
coder 83 which, in accordance with an adaptive dis- 
crete cosine transformation (ADCT), encodes the lu- 30 
minance signal Y that has entered from the pre-proc- 
essor 81 and the color-difference signals Cr', Cb' that 
have entered from the sampling unit 82. 
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Fig. 21 is a block diagram illustrating the con- 
struction of the decoder 104. 

As shown in Fig. 21, the decoder 104 includes a 
receiving circuit 51 which, upon receiving a code or ao 
the like from a communication line, etc., separates 
the code into an arithmetic code and a Huffman code 
by a flag. 

An arithmetic decoding circuit 52 decodes the ar- 
ithmetic code, which has entered from the receiving 45 
circuit 51, through a procedure which is the reverse 
of that of the arithmetic encoding circuit 255, and out- 
puts color-character data. A reverse orthogonal 
transformation circuit 53 first decodes the Huffman 
code through a procedure which is the reverse of that 50 
of the orthogonal-transformation encoding circuit 
257, and then applies reverse orthogonal transfor- 
mation processing to the decoded result, thereby out- 
putting image data. A smoothing circuit 54 smoothes 
block distortion of the image data that has entered 55 
from the reverse orthogonal transformation circuit 
53. A combining circuit 55 combines the color-charac- 
ter data from the arithmetic decoding circuit 52 and 



the i mage data from the smoothing circuit 54 to repro- 
duce the image data. It should be noted that the com- 
bining circuit 55 combines the result of multiplying the 
R, G, B values of color-character data by respective 
ones of prescribed coefficients, whereby priority is 
given to color- character data so that clear color char- 
acters can be reproduced. 

In this embodiment, the reason why smoothing is 
applied to the image data outputted by the reverse or- 
thogonal transformation circuit 53 but not to the color- 
character data outputted by the arithmetic decoding 
circuit 52 is that this would cause a decline in the re- 
solution of the character. 

In accordance with the present invention as de- 
scribed above, effects similar to those of the first em- 
bodiment are obtained. Moreover, the accuracy at 
which color characters are detected is improved by 
detecting color-character portions in character areas. 

Further, in accordance with this embodiment 
separated color-character data is arithmetically en- 
coded, a portion of data from which a color character 
has been separated and removed is replaced by an 
average value of the surrounding peripheral pixels 
and the image data from which the color-character 
data has been removed is encoded by orthogonal 
transformation. As a result, efficient encoding of im- 
age data can be realized without causing a decline in 
picture quality. 

The present embodiment is not limited to an im- 
age communication apparatus such as a color facsi- 
mile. For example, as illustrated in Fig. 22, a memory 
106 may be provided instead of a communication line 
so that the invention can be applied also to an image 
storage apparatus such as an image filing apparatus. 
It should be noted that the memory 106 may be a hard 
disk, a semiconductor memory or an optomagnetic 
disk and is capable of storing a number of images. An 
arrangement may be adopted in which arithmetic 
codes of color-character data and Huffman codes of 
image data are stored individually or stored collec- 
tively image by image. 

Furthermore, in the image memory apparatus of 
Fig. 22, it is possible to reproduce only color charac- 
ters or only images other than characters, by way of 
example. 

The present invention can be applied to a system, 
such as a reader or printer system, constituted by a 
plurality of devices or to an apparatus comprising a 
single device. Furthermore, it goes without saying 
that the invention is applicable also to a case where 
the object of the invention is attained by supplying a 
program to a system or apparatus. 

Thus, in accordance with the present invention, 
it is possible to provide an image processing method 
and apparatus for extracting edge information in im- 
age data from at least one of a plurality of items of 
band-limited image data produced from the image 
data, and detecting character areas in the image data 
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based upon the distribution of the edge information. 

Further, in accordance with the present inven- 
tion, it is possible to provide an image processing ap- 
paratus for extracting edge information in image data 
from at least one of a plurality of items of band-limited 
image data produced from the image data, detecting 
character areas in the image data based upon the dis- 
tribution of the edge information, separating the im- 
age data into character data and non-character data 
based upon the character-area information detected 
and encoding the separated character data and non- 
character data. 

It should be noted that the above-mentioned 
band-limited image data can be realized not only by 
a wavelet transformation but also by other methods 
such as window Fourier transform. 

Further, though image discrimination may be 
performed in either block units or pixel units, better 
information is obtained using pixel units. 

In addition, the manner in which bands are ob- 
tained is not limited to that of the foregoing embodi- 
ments. Furthermore, the characteristic of an image 
are not limited to whether or not an image is a char- 
acter or not. For example, such characteristic may be 
whether or not an image is a dot image or not 

As many apparently widely different embodi- 
ments of the present invention can be made without 
departing from the spirit and scope thereof, it is to be 
understood that the invention is not limited to the spe- 
cific embodiments thereof except as defined in the 
appended claims. 



Claims 

1. An image processing apparatus comprising: 

, input means for entering image data; 

conversion means for converting the im- 
age data into a plurality of band-limited image 
data by using a bandpass fitter; and 

discrimination means for discriminating a 
characteristic of an image, which is represented 
by the image data, on the basis of at least one of 
the plurality of band-limited image data. 

45 

2. The apparatus according to claim 1 , wherein said 
conversion means performs a two-dimensional 
wavelet transformation. 

3. The apparatus according to claim 1 , wherein said 50 
conversion means comprises a low-pass filter 

and a high-pass filter. 

4. The apparatus according to claim 1 , wherein said 
band-limited image data represents an edge 55 
component 

5. The apparatus according to claim 4, wherein said 



discrimination means discriminates an image 
characteristic using said edge component. 

6. The apparatus according to claim 1 . further conrv 



12. An image processing method comprising the 
steps of: 

entering image data; 

classifying the image data in terms of fre- 
quency; and 

determining characteristics of the image 
represented by the data based on the frequency 
classification. 



5 prising correcting means for correcting results of 

said discrimination. 

7. The apparatus according to claim 1 , further com- 
prising encoding means for encoding the image 

70 data, which has been entered by said input 

means, in conformity with results of said discrim- 
ination. 

8. The apparatus according to claim 7, wherein said 
15 encoding means has a first encoding circuit for 

encoding a character area and a second encod- 
ing circuit for encoding other areas. 

9. The apparatus according to claim 8, wherein said 
20 first encoding circuit performs entropy encoding 

and said second encoding circuit performs infor- 
mation non- preserving coding. 

10. An image processing method comprising the 
25 steps of: 

entering image data; 

converting the image data into a plurality 
of band-limited image data by using a band-pass 
filter, and 

30 discriminating a characteristic of an im- 

age, which is represented by the image data, on 
the basis of at least one of the plurality of band- 
limited image data. 

35 11. An image processing apparatus comprising: 
means for entering image data; 
means for classifying the image data in 
terms of frequency; and 

means for determining characteristics of 
40 the image represented by the data based on the 

frequency classification. 
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FIG. 12 
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